Faults-slip-through - a concept for measuring the efficiency of the test process

SUMMARY In market-driven development where time-to-market is of crucial importance, software development companies seek improvements that can decrease the lead-time and improve the delivery precision. One way to achieve this is by analyzing the test process since rework commonly accounts for more than half of the development time. A large reason for high rework costs is fault slippages from earlier phases where they are cheaper to find and remove. As input to improvements, this paper introduces a measure that can quantify this relationship. That is, a measure called faults-slip-through, which determines which faults that would have been more cost-effective to find in an earlier phase. The method presented in this paper also determines the excessive costs of the faults that slipped through phases, i.e. the improvement potential. The method was validated through practical application in two software development projects at the telecom company Ericsson AB. The results of the case study determined that the implementation phase had the largest improvement potential in the two studied projects since it caused a large faults-slip-through to later phases, i.e. 85 and 86 percent of the total improvement potential of each project.